IMPORTS¶

In [24]:
import pandas as pd
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
import seaborn as sns
import datetime as dt
import sklearn

from prophet import Prophet
from scipy.stats import mode

import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
pd.set_option('display.max_columns', None)

Second Dataset¶

The second dataset, METEO, contains hourly information from The Weather Company weather stations in the area of influence in the period from 29-06-2015 to 30-06-2022, it has the following variables:

• validTimeUtc: Date
• precip1Hour: Rain volume in the last hour
• precip6Hour: Rain volume in the last 6 hours
• precip24Hour: Rain volume in the last 24 hours
• precip2Day: Rain volume in the last 2 days
• precip3Day: Rain volume in the last 3 days
• precip7Day: Rain volume in the last 7 days
• precipMtd: Rain volume in the current month
• precipYtd: Rain volume in the current year
• pressureChange: Maximum variation in atmospheric pressure in the last 3 hours
• pressureMeanSeaLevel: Barometric difference with respect to sea level
• relativeHumidity: Relative humidity
• snow1Hour: Snow volume in the last hour
• snow6Hour: Snow volume in the last 6 hours
• snow24Hour: Snow volume in the last 24 hours
• snow2Day: Snow volume in the last 2 days
• snow3Day: Snow volume in the last 3 days
• snow7Day: Snow volume in the last 7 days
• snowMtd: Snow volume in the current month
• snowSeason: Quarterly snow volume (DIC-FEB, MAR-MAY, JUN-AGO, SEP-NOV)
• snowYtd: Snow volume in the current year
• temperature: Ambient temperature at 2 meters above the ground
• temperatureChange24Hour: Temperature variation from previous day
• temperatureMax24Hour: Maximum temperature last 24 hours
• temperatureMin24Hour: Minimum temperature last 24 hours
• temperatureDewPoint: DewPoint, temperature at which the air must be cooled at constant pressure to reach saturation. The dew point is also an indirect measure of air humidity.
• temperatureFeelsLike: Thermal sensation. Apparent temperature resulting from a combination of temperature, humidity and wind speed.
• uvIndex: Ultraviolet radiation categorized: -2, -1 = not available / 0-2 = low / 3-5 = moderate / 6-7 = high / 8-10 = very high / 11-16 = extreme.
• visibility: Horizontal visibility from the weather station, 999 equals unlimited
• windDirection: Wind Direction in degrees 0 - North, 90 - East, 180 - South, 270 - West
• windGust: Maximum wind gust velocity recorded during the observation period
• windSpeed: Wind force
• ID_ESTACION: Weather station identifier

DATA PREPARATION¶

Import and brief exploration of the dataset

In [2]:
METEO = pd.read_csv('./Datos_Originales/DATOS_METEO.TXT',delimiter = '|', na_values = 'NA', encoding='UTF-8', parse_dates=['validTimeUtc'])
In [3]:
METEO.head()
Out[3]:
validTimeUtc precip1Hour precip6Hour precip24Hour precip2Day precip3Day precip7Day precipMtd precipYtd pressureChange pressureMeanSeaLevel relativeHumidity snow1Hour snow6Hour snow24Hour snow2Day snow3Day snow7Day snowMtd snowSeason snowYtd temperature temperatureChange24Hour temperatureMax24Hour temperatureMin24Hour temperatureDewPoint temperatureFeelsLike uvIndex visibility windDirection windGust windSpeed ID_ESTACION
0 2015-06-29 16:20:00 0.0 0.0 0.0 NaN NaN NaN NaN NaN -1.4 NaN 27.0 0.0 0.0 0.0 NaN NaN NaN NaN NaN NaN 34.5 -1.1 36.3 17.9 12.8 34.5 2.0 16.09 NaN NaN 18.7 13
1 2015-06-29 17:20:00 0.0 0.0 0.0 NaN NaN NaN NaN NaN -1.0 NaN 26.3 0.0 0.0 0.0 NaN NaN NaN NaN NaN NaN 34.3 -0.4 35.0 17.9 12.3 34.3 1.0 16.09 NaN NaN 18.0 13
2 2015-06-29 18:20:00 0.0 0.0 0.0 NaN NaN NaN NaN NaN -0.3 NaN 29.0 0.0 0.0 0.0 NaN NaN NaN NaN NaN NaN 32.8 0.1 34.7 17.9 12.4 32.8 0.0 16.09 NaN NaN 16.6 13
3 2015-06-29 19:20:00 0.0 0.0 0.0 NaN NaN NaN NaN NaN 0.3 NaN 33.2 0.0 0.0 0.0 NaN NaN NaN NaN NaN NaN 31.0 0.3 34.7 17.9 12.9 31.0 0.0 16.09 NaN NaN 15.1 13
4 2015-06-29 20:20:00 0.0 0.0 0.0 NaN NaN NaN NaN NaN 0.9 NaN 43.1 0.0 0.0 0.0 NaN NaN NaN NaN NaN NaN 27.6 0.0 34.7 17.9 13.9 28.0 0.0 16.09 NaN NaN 10.1 13
In [4]:
METEO.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 1223660 entries, 0 to 1223659
Data columns (total 33 columns):
 #   Column                   Non-Null Count    Dtype         
---  ------                   --------------    -----         
 0   validTimeUtc             1223660 non-null  datetime64[ns]
 1   precip1Hour              1223640 non-null  float64       
 2   precip6Hour              1223140 non-null  float64       
 3   precip24Hour             1223540 non-null  float64       
 4   precip2Day               750600 non-null   float64       
 5   precip3Day               750600 non-null   float64       
 6   precip7Day               750600 non-null   float64       
 7   precipMtd                750600 non-null   float64       
 8   precipYtd                750600 non-null   float64       
 9   pressureChange           1223640 non-null  float64       
 10  pressureMeanSeaLevel     867520 non-null   float64       
 11  relativeHumidity         1223660 non-null  float64       
 12  snow1Hour                1223640 non-null  float64       
 13  snow6Hour                1223140 non-null  float64       
 14  snow24Hour               1223540 non-null  float64       
 15  snow2Day                 750600 non-null   float64       
 16  snow3Day                 750600 non-null   float64       
 17  snow7Day                 750600 non-null   float64       
 18  snowMtd                  750600 non-null   float64       
 19  snowSeason               750600 non-null   float64       
 20  snowYtd                  750600 non-null   float64       
 21  temperature              1223640 non-null  float64       
 22  temperatureChange24Hour  1223280 non-null  float64       
 23  temperatureMax24Hour     1223560 non-null  float64       
 24  temperatureMin24Hour     1223560 non-null  float64       
 25  temperatureDewPoint      1223640 non-null  float64       
 26  temperatureFeelsLike     1223640 non-null  float64       
 27  uvIndex                  1223640 non-null  float64       
 28  visibility               1223640 non-null  float64       
 29  windDirection            867520 non-null   float64       
 30  windGust                 125679 non-null   float64       
 31  windSpeed                1223660 non-null  float64       
 32  ID_ESTACION              1223660 non-null  int64         
dtypes: datetime64[ns](1), float64(31), int64(1)
memory usage: 308.1 MB

We separate the validTimeUtc variable into "Year", "Month", "Day" and "Time" for future study.

In [5]:
METEO['Year'] = list(map(str,METEO.validTimeUtc.dt.year))
METEO['Month'] = list(map(str,METEO.validTimeUtc.dt.month))
METEO['Day'] = list(map(str,METEO.validTimeUtc.dt.day))
METEO['Time'] = list(METEO.validTimeUtc.dt.time)

We keep the columns we are interested in, i.e. those with hourly data.

In [6]:
columns = ['validTimeUtc', 'Year', 'Month', 'Day', 'Time', 'precip1Hour', 'pressureChange', 'pressureMeanSeaLevel', 'relativeHumidity', 
           'snow1Hour', 'temperature', 'temperatureChange24Hour', 'temperatureMax24Hour', 'temperatureMin24Hour', 'temperatureDewPoint',
           'temperatureFeelsLike', 'uvIndex', 'visibility', 'windSpeed', 'windDirection', 'ID_ESTACION'] 

METEO = METEO[[i for i in METEO.columns if i in columns]].copy(deep = True)
METEO['uvIndex'].replace([-2,-1], np.nan, inplace = True)
METEO
Out[6]:
validTimeUtc precip1Hour pressureChange pressureMeanSeaLevel relativeHumidity snow1Hour temperature temperatureChange24Hour temperatureMax24Hour temperatureMin24Hour temperatureDewPoint temperatureFeelsLike uvIndex visibility windDirection windSpeed ID_ESTACION Year Month Day Time
0 2015-06-29 16:20:00 0.0 -1.4 NaN 27.0 0.0 34.5 -1.1 36.3 17.9 12.8 34.5 2.0 16.09 NaN 18.7 13 2015 6 29 16:20:00
1 2015-06-29 17:20:00 0.0 -1.0 NaN 26.3 0.0 34.3 -0.4 35.0 17.9 12.3 34.3 1.0 16.09 NaN 18.0 13 2015 6 29 17:20:00
2 2015-06-29 18:20:00 0.0 -0.3 NaN 29.0 0.0 32.8 0.1 34.7 17.9 12.4 32.8 0.0 16.09 NaN 16.6 13 2015 6 29 18:20:00
3 2015-06-29 19:20:00 0.0 0.3 NaN 33.2 0.0 31.0 0.3 34.7 17.9 12.9 31.0 0.0 16.09 NaN 15.1 13 2015 6 29 19:20:00
4 2015-06-29 20:20:00 0.0 0.9 NaN 43.1 0.0 27.6 0.0 34.7 17.9 13.9 28.0 0.0 16.09 NaN 10.1 13 2015 6 29 20:20:00
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
1223655 2022-06-30 19:20:00 0.0 2.7 1011.9 46.7 0.0 25.1 0.1 32.3 15.1 12.9 25.1 0.0 13.55 110.0 10.8 8 2022 6 30 19:20:00
1223656 2022-06-30 20:20:00 0.0 3.7 1013.1 57.1 0.0 22.9 0.2 32.3 15.1 13.9 22.9 0.0 13.59 80.0 9.0 8 2022 6 30 20:20:00
1223657 2022-06-30 21:20:00 0.0 3.8 1014.4 65.7 0.0 21.0 0.1 32.3 15.1 14.4 21.0 0.0 13.85 80.0 7.9 8 2022 6 30 21:20:00
1223658 2022-06-30 22:20:00 0.0 3.1 1015.1 75.5 0.0 19.9 0.1 32.3 15.1 15.4 19.9 0.0 13.17 50.0 8.3 8 2022 6 30 22:20:00
1223659 2022-06-30 23:20:00 0.0 2.0 1015.3 90.9 0.0 18.7 -0.2 32.3 15.1 17.2 18.7 0.0 9.51 50.0 10.4 8 2022 6 30 23:20:00

1223660 rows × 21 columns

In [7]:
METEO.describe()
Out[7]:
validTimeUtc precip1Hour pressureChange pressureMeanSeaLevel relativeHumidity snow1Hour temperature temperatureChange24Hour temperatureMax24Hour temperatureMin24Hour temperatureDewPoint temperatureFeelsLike uvIndex visibility windDirection windSpeed ID_ESTACION
count 1223660 1.223640e+06 1.223640e+06 867520.000000 1.223660e+06 1.223640e+06 1.223640e+06 1.223280e+06 1.223560e+06 1.223560e+06 1.223640e+06 1.223640e+06 1.223640e+06 1.223640e+06 867520.000000 1.223660e+06 1.223660e+06
mean 2019-01-01 13:32:01.906412032 6.657947e-02 -4.230329e-03 1017.788293 6.691634e+01 7.397601e-04 1.542311e+01 -4.719181e-02 2.173558e+01 1.006881e+01 8.223297e+00 1.474654e+01 1.354013e+00 1.286694e+01 184.346724 1.049545e+01 9.500000e+00
min 2015-06-29 16:20:00 0.000000e+00 -6.200000e+00 988.600000 6.500000e+00 0.000000e+00 -7.900000e+00 -1.040000e+01 1.000000e-01 -7.900000e+00 -1.760000e+01 -1.130000e+01 0.000000e+00 1.000000e-01 0.000000 0.000000e+00 0.000000e+00
25% 2017-04-03 11:20:00 0.000000e+00 -8.000000e-01 1014.000000 4.960000e+01 0.000000e+00 9.300000e+00 -8.000000e-01 1.510000e+01 5.300000e+00 4.100000e+00 7.900000e+00 0.000000e+00 1.211000e+01 70.000000 5.400000e+00 4.750000e+00
50% 2019-01-02 12:20:00 0.000000e+00 0.000000e+00 1017.500000 6.840000e+01 0.000000e+00 1.470000e+01 1.000000e-01 2.100000e+01 9.700000e+00 8.200000e+00 1.420000e+01 0.000000e+00 1.348000e+01 210.000000 8.600000e+00 9.500000e+00
75% 2020-10-01 09:20:00 0.000000e+00 8.000000e-01 1021.800000 8.600000e+01 0.000000e+00 2.090000e+01 8.000000e-01 2.840000e+01 1.530000e+01 1.290000e+01 2.090000e+01 2.000000e+00 1.458000e+01 270.000000 1.400000e+01 1.425000e+01
max 2022-06-30 23:20:00 6.030000e+01 5.700000e+00 1037.000000 1.000000e+02 5.000000e+00 4.320000e+01 8.900000e+00 4.320000e+01 2.800000e+01 2.330000e+01 4.320000e+01 1.000000e+01 1.609000e+01 350.000000 5.870000e+01 1.900000e+01
std NaN 5.892155e-01 1.129044e+00 6.363315 2.212223e+01 3.520595e-02 8.007426e+00 1.509542e+00 7.943553e+00 6.080957e+00 6.036721e+00 8.808097e+00 2.354698e+00 2.875003e+00 101.357652 6.858221e+00 5.766284e+00

EDA¶

In [8]:
columns_with_missing_data_METEO = METEO.isnull().sum()[METEO.isnull().sum() > 0].index
METEO.groupby('Year')[columns_with_missing_data_METEO].apply(lambda x: x.isnull().sum())
Out[8]:
precip1Hour pressureChange pressureMeanSeaLevel snow1Hour temperature temperatureChange24Hour temperatureMax24Hour temperatureMin24Hour temperatureDewPoint temperatureFeelsLike uvIndex visibility windDirection
Year
2015 0 0 87760 0 0 140 100 100 0 0 0 0 87760
2016 20 20 174020 20 20 20 0 0 20 20 20 20 174020
2017 0 0 94360 0 0 100 0 0 0 0 0 0 94360
2018 0 0 0 0 0 60 0 0 0 0 0 0 0
2019 0 0 0 0 0 60 0 0 0 0 0 0 0
2020 0 0 0 0 0 0 0 0 0 0 0 0 0
2021 0 0 0 0 0 0 0 0 0 0 0 0 0
2022 0 0 0 0 0 0 0 0 0 0 0 0 0
In [9]:
sns.heatmap(METEO[columns_with_missing_data_METEO].isnull(), cbar=False);
In [10]:
fig, ax = plt.subplots(figsize=(28, 10))
g = sns.barplot(y=METEO[columns_with_missing_data_METEO].columns, x=METEO[columns_with_missing_data_METEO].isnull().sum(), orient = 'h', palette = 'Blues')
In [11]:
sns.set_theme(style="white")

g = sns.PairGrid(METEO, diag_sharey=False)
g.map_upper(sns.scatterplot, s=15)
g.map_lower(sns.scatterplot, s=15)
g.map_diag(sns.kdeplot, lw=2)
plt.show()
In [12]:
METEO.columns
Out[12]:
Index(['validTimeUtc', 'precip1Hour', 'pressureChange', 'pressureMeanSeaLevel',
       'relativeHumidity', 'snow1Hour', 'temperature',
       'temperatureChange24Hour', 'temperatureMax24Hour',
       'temperatureMin24Hour', 'temperatureDewPoint', 'temperatureFeelsLike',
       'uvIndex', 'visibility', 'windDirection', 'windSpeed', 'ID_ESTACION',
       'Year', 'Month', 'Day', 'Time'],
      dtype='object')
In [13]:
correlation_matrix = METEO[['precip1Hour', 'pressureChange', 'pressureMeanSeaLevel',
                            'relativeHumidity', 'snow1Hour', 'temperature', 
                            'temperatureChange24Hour', 'temperatureMax24Hour',
                            'temperatureMin24Hour', 'temperatureDewPoint', 'temperatureFeelsLike',
                            'uvIndex', 'visibility', 'windDirection', 'windSpeed']].corr()

sns.heatmap(correlation_matrix, annot=True, cmap='Blues', annot_kws={"size": 8})
plt.show()
In [14]:
columns = METEO.columns
columns = columns.drop(['validTimeUtc', 'Year', 'Month', 'Day', 'Time', 'ID_ESTACION'])

for col in columns:
    plt.figure(figsize=(8,4))
    sns.histplot(x=METEO[col], kde=True)
    plt.title(col)
    plt.show()

DATA PREPROCESSING¶

In [15]:
METEO_MISS = METEO.groupby('Year')[columns_with_missing_data_METEO].apply(lambda x: x.isnull().sum())
for i, row in METEO_MISS.iterrows():
    METEO_MISS.loc[i,'TAMAÑO'] = len(METEO[METEO.Year == i])

METEO_MISS
Out[15]:
precip1Hour pressureChange pressureMeanSeaLevel snow1Hour temperature temperatureChange24Hour temperatureMax24Hour temperatureMin24Hour temperatureDewPoint temperatureFeelsLike uvIndex visibility windDirection TAMAÑO
Year
2015 0 0 87760 0 0 140 100 100 0 0 0 0 87760 87760.0
2016 20 20 174020 20 20 20 0 0 20 20 20 20 174020 174020.0
2017 0 0 94360 0 0 100 0 0 0 0 0 0 94360 174360.0
2018 0 0 0 0 0 60 0 0 0 0 0 0 0 174960.0
2019 0 0 0 0 0 60 0 0 0 0 0 0 0 175060.0
2020 0 0 0 0 0 0 0 0 0 0 0 0 0 175540.0
2021 0 0 0 0 0 0 0 0 0 0 0 0 0 175100.0
2022 0 0 0 0 0 0 0 0 0 0 0 0 0 86860.0

We are going to treat this dataset as a time series since it is easy to see it this way thanks to the temporal variable "validTimeUtc" and we are going to try to predict the missing data with the Meta (Facebook) library, Prophet, because Prophet usually predicts future data with data from the past and the missing data we have are from the past, we are going to try to "cheat" Prophet by changing the order dates and then we will see the results of the predictions.

In [16]:
%%time
METEO['fechas_invertida'] = METEO['validTimeUtc'][::-1].reset_index(drop=True)

grupos = METEO.groupby('ID_ESTACION')

dataframes = {}
for estacion, grupo in grupos:
    dataframes[estacion] = grupo.drop(grupo.index[0]).reset_index(level=0, drop=True)
    
dataframes_completos = {}

for estacion, df in dataframes.items():
    columns_aus = df.columns[df.isnull().any()].tolist()
    df_completo = df.copy(deep = True)
    
    for col in columns_aus:
        index_fechas_faltantes = df[df[col].isnull()].index.tolist()
        df_prophet_train = df[~df[col].isnull()].rename(columns={'fechas_invertida': 'ds', col: 'y'})[['ds', 'y']]
        predict = df[df[col].isnull()].rename(columns={'fechas_invertida': 'ds', col: 'y'})[['ds', 'y']]
        model = Prophet()
        model.fit(df_prophet_train)

        if predict.empty:
            pass
        else:
            forecast = model.predict(predict)

        df_completo[col] = df_completo[col].fillna(df_completo['fechas_invertida'].map(forecast.set_index('ds')['yhat']))
        fig = model.plot(forecast)
        
    dataframes_completos[estacion] = df_completo.drop(df_completo.index[0]).reset_index(level=0, drop=True)

METEO_PROPHET = pd.concat(dataframes_completos.values(), ignore_index=True)
METEO_PROPHET = METEO_PROPHET.drop(['fechas_invertida'], axis = 1)

16:11:41 - cmdstanpy - INFO - Chain [1] start processing
16:11:54 - cmdstanpy - INFO - Chain [1] done processing
16:11:57 - cmdstanpy - INFO - Chain [1] start processing
16:12:04 - cmdstanpy - INFO - Chain [1] done processing
16:12:07 - cmdstanpy - INFO - Chain [1] start processing
16:12:27 - cmdstanpy - INFO - Chain [1] done processing
16:12:33 - cmdstanpy - INFO - Chain [1] start processing
16:12:38 - cmdstanpy - INFO - Chain [1] done processing
16:12:41 - cmdstanpy - INFO - Chain [1] start processing
16:12:56 - cmdstanpy - INFO - Chain [1] done processing
16:12:59 - cmdstanpy - INFO - Chain [1] start processing
16:13:03 - cmdstanpy - INFO - Chain [1] done processing
16:13:06 - cmdstanpy - INFO - Chain [1] start processing
16:13:27 - cmdstanpy - INFO - Chain [1] done processing
16:13:30 - cmdstanpy - INFO - Chain [1] start processing
16:13:45 - cmdstanpy - INFO - Chain [1] done processing
16:13:48 - cmdstanpy - INFO - Chain [1] start processing
16:14:12 - cmdstanpy - INFO - Chain [1] done processing
16:14:15 - cmdstanpy - INFO - Chain [1] start processing
16:14:33 - cmdstanpy - INFO - Chain [1] done processing
16:14:36 - cmdstanpy - INFO - Chain [1] start processing
16:14:40 - cmdstanpy - INFO - Chain [1] done processing
16:14:43 - cmdstanpy - INFO - Chain [1] start processing
16:14:59 - cmdstanpy - INFO - Chain [1] done processing
16:15:01 - cmdstanpy - INFO - Chain [1] start processing
16:15:06 - cmdstanpy - INFO - Chain [1] done processing
16:15:12 - cmdstanpy - INFO - Chain [1] start processing
16:15:19 - cmdstanpy - INFO - Chain [1] done processing
16:15:22 - cmdstanpy - INFO - Chain [1] start processing
16:15:30 - cmdstanpy - INFO - Chain [1] done processing
16:15:33 - cmdstanpy - INFO - Chain [1] start processing
16:15:51 - cmdstanpy - INFO - Chain [1] done processing
16:15:57 - cmdstanpy - INFO - Chain [1] start processing
16:16:06 - cmdstanpy - INFO - Chain [1] done processing
16:16:09 - cmdstanpy - INFO - Chain [1] start processing
16:16:23 - cmdstanpy - INFO - Chain [1] done processing
16:16:26 - cmdstanpy - INFO - Chain [1] start processing
16:16:29 - cmdstanpy - INFO - Chain [1] done processing
16:16:32 - cmdstanpy - INFO - Chain [1] start processing
16:17:00 - cmdstanpy - INFO - Chain [1] done processing
16:17:03 - cmdstanpy - INFO - Chain [1] start processing
16:17:31 - cmdstanpy - INFO - Chain [1] done processing
C:\Users\Usuario\anaconda3\Lib\site-packages\prophet\plot.py:67: RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`). Consider using `matplotlib.pyplot.close()`.
  fig = plt.figure(facecolor='w', figsize=figsize)
16:17:34 - cmdstanpy - INFO - Chain [1] start processing
16:17:51 - cmdstanpy - INFO - Chain [1] done processing
16:17:54 - cmdstanpy - INFO - Chain [1] start processing
16:18:14 - cmdstanpy - INFO - Chain [1] done processing
16:18:17 - cmdstanpy - INFO - Chain [1] start processing
16:18:21 - cmdstanpy - INFO - Chain [1] done processing
16:18:24 - cmdstanpy - INFO - Chain [1] start processing
16:18:41 - cmdstanpy - INFO - Chain [1] done processing
16:18:43 - cmdstanpy - INFO - Chain [1] start processing
16:18:48 - cmdstanpy - INFO - Chain [1] done processing
16:18:54 - cmdstanpy - INFO - Chain [1] start processing
16:19:03 - cmdstanpy - INFO - Chain [1] done processing
16:19:06 - cmdstanpy - INFO - Chain [1] start processing
16:19:13 - cmdstanpy - INFO - Chain [1] done processing
16:19:16 - cmdstanpy - INFO - Chain [1] start processing
16:19:38 - cmdstanpy - INFO - Chain [1] done processing
16:19:44 - cmdstanpy - INFO - Chain [1] start processing
16:19:51 - cmdstanpy - INFO - Chain [1] done processing
16:19:54 - cmdstanpy - INFO - Chain [1] start processing
16:20:06 - cmdstanpy - INFO - Chain [1] done processing
16:20:09 - cmdstanpy - INFO - Chain [1] start processing
16:20:14 - cmdstanpy - INFO - Chain [1] done processing
16:20:17 - cmdstanpy - INFO - Chain [1] start processing
16:20:38 - cmdstanpy - INFO - Chain [1] done processing
16:20:40 - cmdstanpy - INFO - Chain [1] start processing
16:21:01 - cmdstanpy - INFO - Chain [1] done processing
16:21:04 - cmdstanpy - INFO - Chain [1] start processing
16:21:24 - cmdstanpy - INFO - Chain [1] done processing
16:21:27 - cmdstanpy - INFO - Chain [1] start processing
16:21:41 - cmdstanpy - INFO - Chain [1] done processing
16:21:44 - cmdstanpy - INFO - Chain [1] start processing
16:21:48 - cmdstanpy - INFO - Chain [1] done processing
16:21:51 - cmdstanpy - INFO - Chain [1] start processing
16:22:05 - cmdstanpy - INFO - Chain [1] done processing
16:22:07 - cmdstanpy - INFO - Chain [1] start processing
16:22:14 - cmdstanpy - INFO - Chain [1] done processing
16:22:20 - cmdstanpy - INFO - Chain [1] start processing
16:22:26 - cmdstanpy - INFO - Chain [1] done processing
16:22:29 - cmdstanpy - INFO - Chain [1] start processing
16:22:37 - cmdstanpy - INFO - Chain [1] done processing
16:22:39 - cmdstanpy - INFO - Chain [1] start processing
16:23:01 - cmdstanpy - INFO - Chain [1] done processing
16:23:08 - cmdstanpy - INFO - Chain [1] start processing
16:23:14 - cmdstanpy - INFO - Chain [1] done processing
16:23:17 - cmdstanpy - INFO - Chain [1] start processing
16:23:36 - cmdstanpy - INFO - Chain [1] done processing
16:23:39 - cmdstanpy - INFO - Chain [1] start processing
16:23:44 - cmdstanpy - INFO - Chain [1] done processing
16:23:47 - cmdstanpy - INFO - Chain [1] start processing
16:24:12 - cmdstanpy - INFO - Chain [1] done processing
16:24:15 - cmdstanpy - INFO - Chain [1] start processing
16:24:30 - cmdstanpy - INFO - Chain [1] done processing
16:24:33 - cmdstanpy - INFO - Chain [1] start processing
16:24:58 - cmdstanpy - INFO - Chain [1] done processing
16:25:01 - cmdstanpy - INFO - Chain [1] start processing
16:25:20 - cmdstanpy - INFO - Chain [1] done processing
16:25:23 - cmdstanpy - INFO - Chain [1] start processing
16:25:27 - cmdstanpy - INFO - Chain [1] done processing
16:25:30 - cmdstanpy - INFO - Chain [1] start processing
16:26:03 - cmdstanpy - INFO - Chain [1] done processing
16:26:05 - cmdstanpy - INFO - Chain [1] start processing
16:26:10 - cmdstanpy - INFO - Chain [1] done processing
16:26:16 - cmdstanpy - INFO - Chain [1] start processing
16:26:19 - cmdstanpy - INFO - Chain [1] done processing
16:26:22 - cmdstanpy - INFO - Chain [1] start processing
16:26:31 - cmdstanpy - INFO - Chain [1] done processing
16:26:34 - cmdstanpy - INFO - Chain [1] start processing
16:26:50 - cmdstanpy - INFO - Chain [1] done processing
16:26:56 - cmdstanpy - INFO - Chain [1] start processing
16:27:05 - cmdstanpy - INFO - Chain [1] done processing
16:27:08 - cmdstanpy - INFO - Chain [1] start processing
16:27:22 - cmdstanpy - INFO - Chain [1] done processing
16:27:25 - cmdstanpy - INFO - Chain [1] start processing
16:27:31 - cmdstanpy - INFO - Chain [1] done processing
16:27:33 - cmdstanpy - INFO - Chain [1] start processing
16:27:58 - cmdstanpy - INFO - Chain [1] done processing
16:28:01 - cmdstanpy - INFO - Chain [1] start processing
16:28:19 - cmdstanpy - INFO - Chain [1] done processing
16:28:22 - cmdstanpy - INFO - Chain [1] start processing
16:28:42 - cmdstanpy - INFO - Chain [1] done processing
16:28:45 - cmdstanpy - INFO - Chain [1] start processing
16:29:02 - cmdstanpy - INFO - Chain [1] done processing
16:29:05 - cmdstanpy - INFO - Chain [1] start processing
16:29:09 - cmdstanpy - INFO - Chain [1] done processing
16:29:12 - cmdstanpy - INFO - Chain [1] start processing
16:29:35 - cmdstanpy - INFO - Chain [1] done processing
16:29:37 - cmdstanpy - INFO - Chain [1] start processing
16:29:41 - cmdstanpy - INFO - Chain [1] done processing
16:29:47 - cmdstanpy - INFO - Chain [1] start processing
16:29:58 - cmdstanpy - INFO - Chain [1] done processing
16:30:01 - cmdstanpy - INFO - Chain [1] start processing
16:30:09 - cmdstanpy - INFO - Chain [1] done processing
16:30:11 - cmdstanpy - INFO - Chain [1] start processing
16:30:31 - cmdstanpy - INFO - Chain [1] done processing
16:30:37 - cmdstanpy - INFO - Chain [1] start processing
16:30:47 - cmdstanpy - INFO - Chain [1] done processing
16:30:49 - cmdstanpy - INFO - Chain [1] start processing
16:31:05 - cmdstanpy - INFO - Chain [1] done processing
16:31:09 - cmdstanpy - INFO - Chain [1] start processing
16:31:11 - cmdstanpy - INFO - Chain [1] done processing
16:31:14 - cmdstanpy - INFO - Chain [1] start processing
16:31:46 - cmdstanpy - INFO - Chain [1] done processing
16:31:49 - cmdstanpy - INFO - Chain [1] start processing
16:32:14 - cmdstanpy - INFO - Chain [1] done processing
16:32:17 - cmdstanpy - INFO - Chain [1] start processing
16:32:42 - cmdstanpy - INFO - Chain [1] done processing
16:32:44 - cmdstanpy - INFO - Chain [1] start processing
16:32:57 - cmdstanpy - INFO - Chain [1] done processing
16:33:00 - cmdstanpy - INFO - Chain [1] start processing
16:33:04 - cmdstanpy - INFO - Chain [1] done processing
16:33:06 - cmdstanpy - INFO - Chain [1] start processing
16:33:28 - cmdstanpy - INFO - Chain [1] done processing
16:33:31 - cmdstanpy - INFO - Chain [1] start processing
16:33:37 - cmdstanpy - INFO - Chain [1] done processing
16:33:43 - cmdstanpy - INFO - Chain [1] start processing
16:33:52 - cmdstanpy - INFO - Chain [1] done processing
16:33:55 - cmdstanpy - INFO - Chain [1] start processing
16:34:05 - cmdstanpy - INFO - Chain [1] done processing
16:34:07 - cmdstanpy - INFO - Chain [1] start processing
16:34:26 - cmdstanpy - INFO - Chain [1] done processing
16:34:31 - cmdstanpy - INFO - Chain [1] start processing
16:34:39 - cmdstanpy - INFO - Chain [1] done processing
16:34:42 - cmdstanpy - INFO - Chain [1] start processing
16:35:01 - cmdstanpy - INFO - Chain [1] done processing
16:35:04 - cmdstanpy - INFO - Chain [1] start processing
16:35:07 - cmdstanpy - INFO - Chain [1] done processing
16:35:10 - cmdstanpy - INFO - Chain [1] start processing
16:35:44 - cmdstanpy - INFO - Chain [1] done processing
16:35:47 - cmdstanpy - INFO - Chain [1] start processing
16:36:11 - cmdstanpy - INFO - Chain [1] done processing
16:36:14 - cmdstanpy - INFO - Chain [1] start processing
16:36:34 - cmdstanpy - INFO - Chain [1] done processing
16:36:37 - cmdstanpy - INFO - Chain [1] start processing
16:36:56 - cmdstanpy - INFO - Chain [1] done processing
16:36:59 - cmdstanpy - INFO - Chain [1] start processing
16:37:04 - cmdstanpy - INFO - Chain [1] done processing
16:37:07 - cmdstanpy - INFO - Chain [1] start processing
16:37:30 - cmdstanpy - INFO - Chain [1] done processing
16:37:32 - cmdstanpy - INFO - Chain [1] start processing
16:37:40 - cmdstanpy - INFO - Chain [1] done processing
16:37:46 - cmdstanpy - INFO - Chain [1] start processing
16:37:57 - cmdstanpy - INFO - Chain [1] done processing
16:38:00 - cmdstanpy - INFO - Chain [1] start processing
16:38:09 - cmdstanpy - INFO - Chain [1] done processing
16:38:11 - cmdstanpy - INFO - Chain [1] start processing
16:38:29 - cmdstanpy - INFO - Chain [1] done processing
16:38:35 - cmdstanpy - INFO - Chain [1] start processing
16:38:41 - cmdstanpy - INFO - Chain [1] done processing
16:38:44 - cmdstanpy - INFO - Chain [1] start processing
16:39:05 - cmdstanpy - INFO - Chain [1] done processing
16:39:08 - cmdstanpy - INFO - Chain [1] start processing
16:39:12 - cmdstanpy - INFO - Chain [1] done processing
16:39:14 - cmdstanpy - INFO - Chain [1] start processing
16:39:41 - cmdstanpy - INFO - Chain [1] done processing
16:39:44 - cmdstanpy - INFO - Chain [1] start processing
16:40:13 - cmdstanpy - INFO - Chain [1] done processing
16:40:16 - cmdstanpy - INFO - Chain [1] start processing
16:40:34 - cmdstanpy - INFO - Chain [1] done processing
16:40:37 - cmdstanpy - INFO - Chain [1] start processing
16:40:54 - cmdstanpy - INFO - Chain [1] done processing
16:40:57 - cmdstanpy - INFO - Chain [1] start processing
16:41:01 - cmdstanpy - INFO - Chain [1] done processing
16:41:04 - cmdstanpy - INFO - Chain [1] start processing
16:41:29 - cmdstanpy - INFO - Chain [1] done processing
16:41:31 - cmdstanpy - INFO - Chain [1] start processing
16:41:39 - cmdstanpy - INFO - Chain [1] done processing
16:41:45 - cmdstanpy - INFO - Chain [1] start processing
16:41:54 - cmdstanpy - INFO - Chain [1] done processing
16:41:57 - cmdstanpy - INFO - Chain [1] start processing
16:42:03 - cmdstanpy - INFO - Chain [1] done processing
16:42:05 - cmdstanpy - INFO - Chain [1] start processing
16:42:29 - cmdstanpy - INFO - Chain [1] done processing
16:42:35 - cmdstanpy - INFO - Chain [1] start processing
16:42:38 - cmdstanpy - INFO - Chain [1] done processing
16:42:41 - cmdstanpy - INFO - Chain [1] start processing
16:42:55 - cmdstanpy - INFO - Chain [1] done processing
16:42:57 - cmdstanpy - INFO - Chain [1] start processing
16:43:00 - cmdstanpy - INFO - Chain [1] done processing
16:43:03 - cmdstanpy - INFO - Chain [1] start processing
16:43:33 - cmdstanpy - INFO - Chain [1] done processing
16:43:36 - cmdstanpy - INFO - Chain [1] start processing
16:43:55 - cmdstanpy - INFO - Chain [1] done processing
16:43:58 - cmdstanpy - INFO - Chain [1] start processing
16:44:23 - cmdstanpy - INFO - Chain [1] done processing
16:44:26 - cmdstanpy - INFO - Chain [1] start processing
16:44:44 - cmdstanpy - INFO - Chain [1] done processing
16:44:46 - cmdstanpy - INFO - Chain [1] start processing
16:44:50 - cmdstanpy - INFO - Chain [1] done processing
16:44:53 - cmdstanpy - INFO - Chain [1] start processing
16:45:11 - cmdstanpy - INFO - Chain [1] done processing
16:45:13 - cmdstanpy - INFO - Chain [1] start processing
16:45:20 - cmdstanpy - INFO - Chain [1] done processing
16:45:26 - cmdstanpy - INFO - Chain [1] start processing
16:45:35 - cmdstanpy - INFO - Chain [1] done processing
16:45:38 - cmdstanpy - INFO - Chain [1] start processing
16:45:44 - cmdstanpy - INFO - Chain [1] done processing
16:45:46 - cmdstanpy - INFO - Chain [1] start processing
16:46:03 - cmdstanpy - INFO - Chain [1] done processing
16:46:08 - cmdstanpy - INFO - Chain [1] start processing
16:46:14 - cmdstanpy - INFO - Chain [1] done processing
16:46:17 - cmdstanpy - INFO - Chain [1] start processing
16:46:35 - cmdstanpy - INFO - Chain [1] done processing
16:46:37 - cmdstanpy - INFO - Chain [1] start processing
16:46:40 - cmdstanpy - INFO - Chain [1] done processing
16:46:44 - cmdstanpy - INFO - Chain [1] start processing
16:47:19 - cmdstanpy - INFO - Chain [1] done processing
16:47:22 - cmdstanpy - INFO - Chain [1] start processing
16:47:47 - cmdstanpy - INFO - Chain [1] done processing
16:47:50 - cmdstanpy - INFO - Chain [1] start processing
16:48:11 - cmdstanpy - INFO - Chain [1] done processing
16:48:14 - cmdstanpy - INFO - Chain [1] start processing
16:48:33 - cmdstanpy - INFO - Chain [1] done processing
16:48:36 - cmdstanpy - INFO - Chain [1] start processing
16:48:40 - cmdstanpy - INFO - Chain [1] done processing
16:48:43 - cmdstanpy - INFO - Chain [1] start processing
16:49:10 - cmdstanpy - INFO - Chain [1] done processing
16:49:12 - cmdstanpy - INFO - Chain [1] start processing
16:49:15 - cmdstanpy - INFO - Chain [1] done processing
16:49:21 - cmdstanpy - INFO - Chain [1] start processing
16:49:33 - cmdstanpy - INFO - Chain [1] done processing
16:49:36 - cmdstanpy - INFO - Chain [1] start processing
16:49:43 - cmdstanpy - INFO - Chain [1] done processing
16:49:45 - cmdstanpy - INFO - Chain [1] start processing
16:50:04 - cmdstanpy - INFO - Chain [1] done processing
16:50:10 - cmdstanpy - INFO - Chain [1] start processing
16:50:17 - cmdstanpy - INFO - Chain [1] done processing
16:50:20 - cmdstanpy - INFO - Chain [1] start processing
16:50:42 - cmdstanpy - INFO - Chain [1] done processing
16:50:45 - cmdstanpy - INFO - Chain [1] start processing
16:50:47 - cmdstanpy - INFO - Chain [1] done processing
16:50:50 - cmdstanpy - INFO - Chain [1] start processing
16:51:12 - cmdstanpy - INFO - Chain [1] done processing
16:51:15 - cmdstanpy - INFO - Chain [1] start processing
16:51:31 - cmdstanpy - INFO - Chain [1] done processing
16:51:34 - cmdstanpy - INFO - Chain [1] start processing
16:51:56 - cmdstanpy - INFO - Chain [1] done processing
16:51:59 - cmdstanpy - INFO - Chain [1] start processing
16:52:14 - cmdstanpy - INFO - Chain [1] done processing
16:52:17 - cmdstanpy - INFO - Chain [1] start processing
16:52:21 - cmdstanpy - INFO - Chain [1] done processing
16:52:24 - cmdstanpy - INFO - Chain [1] start processing
16:52:48 - cmdstanpy - INFO - Chain [1] done processing
16:52:50 - cmdstanpy - INFO - Chain [1] start processing
16:52:56 - cmdstanpy - INFO - Chain [1] done processing
16:53:02 - cmdstanpy - INFO - Chain [1] start processing
16:53:10 - cmdstanpy - INFO - Chain [1] done processing
16:53:13 - cmdstanpy - INFO - Chain [1] start processing
16:53:21 - cmdstanpy - INFO - Chain [1] done processing
16:53:23 - cmdstanpy - INFO - Chain [1] start processing
16:53:41 - cmdstanpy - INFO - Chain [1] done processing
16:53:47 - cmdstanpy - INFO - Chain [1] start processing
16:53:53 - cmdstanpy - INFO - Chain [1] done processing
16:53:56 - cmdstanpy - INFO - Chain [1] start processing
16:54:15 - cmdstanpy - INFO - Chain [1] done processing
16:54:18 - cmdstanpy - INFO - Chain [1] start processing
16:54:23 - cmdstanpy - INFO - Chain [1] done processing
16:54:26 - cmdstanpy - INFO - Chain [1] start processing
16:54:47 - cmdstanpy - INFO - Chain [1] done processing
16:54:50 - cmdstanpy - INFO - Chain [1] start processing
16:55:05 - cmdstanpy - INFO - Chain [1] done processing
16:55:07 - cmdstanpy - INFO - Chain [1] start processing
16:55:30 - cmdstanpy - INFO - Chain [1] done processing
16:55:33 - cmdstanpy - INFO - Chain [1] start processing
16:55:46 - cmdstanpy - INFO - Chain [1] done processing
16:55:49 - cmdstanpy - INFO - Chain [1] start processing
16:55:53 - cmdstanpy - INFO - Chain [1] done processing
16:55:56 - cmdstanpy - INFO - Chain [1] start processing
16:56:10 - cmdstanpy - INFO - Chain [1] done processing
16:56:12 - cmdstanpy - INFO - Chain [1] start processing
16:56:19 - cmdstanpy - INFO - Chain [1] done processing
16:56:25 - cmdstanpy - INFO - Chain [1] start processing
16:56:33 - cmdstanpy - INFO - Chain [1] done processing
16:56:36 - cmdstanpy - INFO - Chain [1] start processing
16:56:43 - cmdstanpy - INFO - Chain [1] done processing
16:56:46 - cmdstanpy - INFO - Chain [1] start processing
16:57:07 - cmdstanpy - INFO - Chain [1] done processing
16:57:12 - cmdstanpy - INFO - Chain [1] start processing
16:57:17 - cmdstanpy - INFO - Chain [1] done processing
16:57:20 - cmdstanpy - INFO - Chain [1] start processing
16:57:39 - cmdstanpy - INFO - Chain [1] done processing
16:57:42 - cmdstanpy - INFO - Chain [1] start processing
16:57:44 - cmdstanpy - INFO - Chain [1] done processing
16:57:47 - cmdstanpy - INFO - Chain [1] start processing
16:58:05 - cmdstanpy - INFO - Chain [1] done processing
16:58:08 - cmdstanpy - INFO - Chain [1] start processing
16:58:23 - cmdstanpy - INFO - Chain [1] done processing
16:58:26 - cmdstanpy - INFO - Chain [1] start processing
16:58:47 - cmdstanpy - INFO - Chain [1] done processing
16:58:50 - cmdstanpy - INFO - Chain [1] start processing
16:59:01 - cmdstanpy - INFO - Chain [1] done processing
16:59:04 - cmdstanpy - INFO - Chain [1] start processing
16:59:08 - cmdstanpy - INFO - Chain [1] done processing
16:59:10 - cmdstanpy - INFO - Chain [1] start processing
16:59:40 - cmdstanpy - INFO - Chain [1] done processing
16:59:42 - cmdstanpy - INFO - Chain [1] start processing
16:59:47 - cmdstanpy - INFO - Chain [1] done processing
16:59:53 - cmdstanpy - INFO - Chain [1] start processing
16:59:58 - cmdstanpy - INFO - Chain [1] done processing
17:00:01 - cmdstanpy - INFO - Chain [1] start processing
17:00:08 - cmdstanpy - INFO - Chain [1] done processing
17:00:10 - cmdstanpy - INFO - Chain [1] start processing
17:00:31 - cmdstanpy - INFO - Chain [1] done processing
17:00:37 - cmdstanpy - INFO - Chain [1] start processing
17:00:43 - cmdstanpy - INFO - Chain [1] done processing
17:00:46 - cmdstanpy - INFO - Chain [1] start processing
17:00:58 - cmdstanpy - INFO - Chain [1] done processing
17:01:01 - cmdstanpy - INFO - Chain [1] start processing
17:01:06 - cmdstanpy - INFO - Chain [1] done processing
17:01:09 - cmdstanpy - INFO - Chain [1] start processing
17:01:38 - cmdstanpy - INFO - Chain [1] done processing
17:01:41 - cmdstanpy - INFO - Chain [1] start processing
17:01:58 - cmdstanpy - INFO - Chain [1] done processing
17:02:01 - cmdstanpy - INFO - Chain [1] start processing
17:02:20 - cmdstanpy - INFO - Chain [1] done processing
17:02:23 - cmdstanpy - INFO - Chain [1] start processing
17:02:37 - cmdstanpy - INFO - Chain [1] done processing
17:02:40 - cmdstanpy - INFO - Chain [1] start processing
17:02:44 - cmdstanpy - INFO - Chain [1] done processing
17:02:47 - cmdstanpy - INFO - Chain [1] start processing
17:03:13 - cmdstanpy - INFO - Chain [1] done processing
17:03:15 - cmdstanpy - INFO - Chain [1] start processing
17:03:22 - cmdstanpy - INFO - Chain [1] done processing
17:03:28 - cmdstanpy - INFO - Chain [1] start processing
17:03:36 - cmdstanpy - INFO - Chain [1] done processing
17:03:40 - cmdstanpy - INFO - Chain [1] start processing
17:03:47 - cmdstanpy - INFO - Chain [1] done processing
17:03:49 - cmdstanpy - INFO - Chain [1] start processing
17:04:04 - cmdstanpy - INFO - Chain [1] done processing
17:04:10 - cmdstanpy - INFO - Chain [1] start processing
17:04:20 - cmdstanpy - INFO - Chain [1] done processing
17:04:23 - cmdstanpy - INFO - Chain [1] start processing
17:04:37 - cmdstanpy - INFO - Chain [1] done processing
17:04:41 - cmdstanpy - INFO - Chain [1] start processing
17:04:44 - cmdstanpy - INFO - Chain [1] done processing
17:04:47 - cmdstanpy - INFO - Chain [1] start processing
17:05:15 - cmdstanpy - INFO - Chain [1] done processing
17:05:18 - cmdstanpy - INFO - Chain [1] start processing
17:05:32 - cmdstanpy - INFO - Chain [1] done processing
17:05:35 - cmdstanpy - INFO - Chain [1] start processing
17:05:53 - cmdstanpy - INFO - Chain [1] done processing
17:05:56 - cmdstanpy - INFO - Chain [1] start processing
17:06:10 - cmdstanpy - INFO - Chain [1] done processing
17:06:14 - cmdstanpy - INFO - Chain [1] start processing
17:06:16 - cmdstanpy - INFO - Chain [1] done processing
17:06:19 - cmdstanpy - INFO - Chain [1] start processing
17:06:38 - cmdstanpy - INFO - Chain [1] done processing
17:06:40 - cmdstanpy - INFO - Chain [1] start processing
17:06:48 - cmdstanpy - INFO - Chain [1] done processing
17:06:54 - cmdstanpy - INFO - Chain [1] start processing
17:07:02 - cmdstanpy - INFO - Chain [1] done processing
17:07:05 - cmdstanpy - INFO - Chain [1] start processing
17:07:14 - cmdstanpy - INFO - Chain [1] done processing
17:07:16 - cmdstanpy - INFO - Chain [1] start processing
17:07:35 - cmdstanpy - INFO - Chain [1] done processing
17:07:41 - cmdstanpy - INFO - Chain [1] start processing
17:07:50 - cmdstanpy - INFO - Chain [1] done processing
17:07:53 - cmdstanpy - INFO - Chain [1] start processing
17:08:11 - cmdstanpy - INFO - Chain [1] done processing
17:08:14 - cmdstanpy - INFO - Chain [1] start processing
17:08:17 - cmdstanpy - INFO - Chain [1] done processing
17:08:20 - cmdstanpy - INFO - Chain [1] start processing
17:08:50 - cmdstanpy - INFO - Chain [1] done processing
17:08:52 - cmdstanpy - INFO - Chain [1] start processing
17:09:09 - cmdstanpy - INFO - Chain [1] done processing
17:09:12 - cmdstanpy - INFO - Chain [1] start processing
17:09:32 - cmdstanpy - INFO - Chain [1] done processing
17:09:36 - cmdstanpy - INFO - Chain [1] start processing
17:09:55 - cmdstanpy - INFO - Chain [1] done processing
17:09:58 - cmdstanpy - INFO - Chain [1] start processing
17:10:02 - cmdstanpy - INFO - Chain [1] done processing
17:10:05 - cmdstanpy - INFO - Chain [1] start processing
17:10:26 - cmdstanpy - INFO - Chain [1] done processing
17:10:28 - cmdstanpy - INFO - Chain [1] start processing
17:10:34 - cmdstanpy - INFO - Chain [1] done processing
17:10:40 - cmdstanpy - INFO - Chain [1] start processing
17:10:48 - cmdstanpy - INFO - Chain [1] done processing
17:10:51 - cmdstanpy - INFO - Chain [1] start processing
17:10:59 - cmdstanpy - INFO - Chain [1] done processing
17:11:02 - cmdstanpy - INFO - Chain [1] start processing
17:11:19 - cmdstanpy - INFO - Chain [1] done processing
17:11:25 - cmdstanpy - INFO - Chain [1] start processing
17:11:34 - cmdstanpy - INFO - Chain [1] done processing
17:11:37 - cmdstanpy - INFO - Chain [1] start processing
17:11:55 - cmdstanpy - INFO - Chain [1] done processing
17:11:58 - cmdstanpy - INFO - Chain [1] start processing
17:12:02 - cmdstanpy - INFO - Chain [1] done processing
17:12:06 - cmdstanpy - INFO - Chain [1] start processing
17:12:35 - cmdstanpy - INFO - Chain [1] done processing
17:12:38 - cmdstanpy - INFO - Chain [1] start processing
17:13:04 - cmdstanpy - INFO - Chain [1] done processing
17:13:07 - cmdstanpy - INFO - Chain [1] start processing
17:13:26 - cmdstanpy - INFO - Chain [1] done processing
17:13:29 - cmdstanpy - INFO - Chain [1] start processing
17:13:44 - cmdstanpy - INFO - Chain [1] done processing
17:13:47 - cmdstanpy - INFO - Chain [1] start processing
17:13:52 - cmdstanpy - INFO - Chain [1] done processing
17:13:55 - cmdstanpy - INFO - Chain [1] start processing
17:14:14 - cmdstanpy - INFO - Chain [1] done processing
17:14:16 - cmdstanpy - INFO - Chain [1] start processing
17:14:21 - cmdstanpy - INFO - Chain [1] done processing
17:14:27 - cmdstanpy - INFO - Chain [1] start processing
17:14:34 - cmdstanpy - INFO - Chain [1] done processing
17:14:37 - cmdstanpy - INFO - Chain [1] start processing
17:14:45 - cmdstanpy - INFO - Chain [1] done processing
17:14:47 - cmdstanpy - INFO - Chain [1] start processing
17:15:07 - cmdstanpy - INFO - Chain [1] done processing
17:15:13 - cmdstanpy - INFO - Chain [1] start processing
17:15:19 - cmdstanpy - INFO - Chain [1] done processing
17:15:22 - cmdstanpy - INFO - Chain [1] start processing
17:15:40 - cmdstanpy - INFO - Chain [1] done processing
17:15:43 - cmdstanpy - INFO - Chain [1] start processing
17:15:45 - cmdstanpy - INFO - Chain [1] done processing
17:15:48 - cmdstanpy - INFO - Chain [1] start processing
17:16:13 - cmdstanpy - INFO - Chain [1] done processing
17:16:16 - cmdstanpy - INFO - Chain [1] start processing
17:16:33 - cmdstanpy - INFO - Chain [1] done processing
17:16:36 - cmdstanpy - INFO - Chain [1] start processing
17:16:56 - cmdstanpy - INFO - Chain [1] done processing
17:16:59 - cmdstanpy - INFO - Chain [1] start processing
17:17:15 - cmdstanpy - INFO - Chain [1] done processing
17:17:18 - cmdstanpy - INFO - Chain [1] start processing
17:17:21 - cmdstanpy - INFO - Chain [1] done processing
17:17:24 - cmdstanpy - INFO - Chain [1] start processing
17:17:45 - cmdstanpy - INFO - Chain [1] done processing
17:17:47 - cmdstanpy - INFO - Chain [1] start processing
17:17:54 - cmdstanpy - INFO - Chain [1] done processing
17:17:59 - cmdstanpy - INFO - Chain [1] start processing
17:18:05 - cmdstanpy - INFO - Chain [1] done processing
17:18:08 - cmdstanpy - INFO - Chain [1] start processing
17:18:14 - cmdstanpy - INFO - Chain [1] done processing
17:18:16 - cmdstanpy - INFO - Chain [1] start processing
17:18:34 - cmdstanpy - INFO - Chain [1] done processing
17:18:40 - cmdstanpy - INFO - Chain [1] start processing
17:18:44 - cmdstanpy - INFO - Chain [1] done processing
17:18:47 - cmdstanpy - INFO - Chain [1] start processing
17:19:02 - cmdstanpy - INFO - Chain [1] done processing
17:19:05 - cmdstanpy - INFO - Chain [1] start processing
17:19:07 - cmdstanpy - INFO - Chain [1] done processing
17:19:10 - cmdstanpy - INFO - Chain [1] start processing
17:19:42 - cmdstanpy - INFO - Chain [1] done processing
17:19:45 - cmdstanpy - INFO - Chain [1] start processing
17:20:03 - cmdstanpy - INFO - Chain [1] done processing
17:20:06 - cmdstanpy - INFO - Chain [1] start processing
17:20:29 - cmdstanpy - INFO - Chain [1] done processing
17:20:32 - cmdstanpy - INFO - Chain [1] start processing
17:20:50 - cmdstanpy - INFO - Chain [1] done processing
17:20:53 - cmdstanpy - INFO - Chain [1] start processing
17:20:56 - cmdstanpy - INFO - Chain [1] done processing
17:21:00 - cmdstanpy - INFO - Chain [1] start processing
17:21:16 - cmdstanpy - INFO - Chain [1] done processing
17:21:18 - cmdstanpy - INFO - Chain [1] start processing
17:21:24 - cmdstanpy - INFO - Chain [1] done processing
17:21:30 - cmdstanpy - INFO - Chain [1] start processing
17:21:41 - cmdstanpy - INFO - Chain [1] done processing
17:21:44 - cmdstanpy - INFO - Chain [1] start processing
17:21:51 - cmdstanpy - INFO - Chain [1] done processing
17:21:53 - cmdstanpy - INFO - Chain [1] start processing
17:22:16 - cmdstanpy - INFO - Chain [1] done processing
17:22:21 - cmdstanpy - INFO - Chain [1] start processing
17:22:29 - cmdstanpy - INFO - Chain [1] done processing
17:22:32 - cmdstanpy - INFO - Chain [1] start processing
17:22:54 - cmdstanpy - INFO - Chain [1] done processing
17:22:57 - cmdstanpy - INFO - Chain [1] start processing
17:23:00 - cmdstanpy - INFO - Chain [1] done processing
17:23:03 - cmdstanpy - INFO - Chain [1] start processing
17:23:31 - cmdstanpy - INFO - Chain [1] done processing
17:23:34 - cmdstanpy - INFO - Chain [1] start processing
17:23:51 - cmdstanpy - INFO - Chain [1] done processing
17:23:55 - cmdstanpy - INFO - Chain [1] start processing
17:24:17 - cmdstanpy - INFO - Chain [1] done processing
17:24:19 - cmdstanpy - INFO - Chain [1] start processing
17:24:34 - cmdstanpy - INFO - Chain [1] done processing
17:24:37 - cmdstanpy - INFO - Chain [1] start processing
17:24:41 - cmdstanpy - INFO - Chain [1] done processing
17:24:44 - cmdstanpy - INFO - Chain [1] start processing
17:25:04 - cmdstanpy - INFO - Chain [1] done processing
17:25:06 - cmdstanpy - INFO - Chain [1] start processing
17:25:12 - cmdstanpy - INFO - Chain [1] done processing

CPU times: total: 3min 54s
Wall time: 1h 13min 37s
In [17]:
METEO_PROPHET
Out[17]:
validTimeUtc precip1Hour pressureChange pressureMeanSeaLevel relativeHumidity snow1Hour temperature temperatureChange24Hour temperatureMax24Hour temperatureMin24Hour temperatureDewPoint temperatureFeelsLike uvIndex visibility windDirection windSpeed ID_ESTACION Year Month Day Time
0 2015-06-29 18:20:00 0.0 -0.1 1013.793693 33.2 0.0 31.9 -0.2 35.5 18.4 13.7 31.9 0.0 16.09 97.653293 15.5 0 2015 6 29 18:20:00
1 2015-06-29 19:20:00 0.0 0.5 1013.778149 39.4 0.0 29.5 -0.1 35.5 18.4 14.3 29.7 0.0 16.09 96.926700 13.7 0 2015 6 29 19:20:00
2 2015-06-29 20:20:00 0.0 1.2 1013.737503 50.9 0.0 25.4 -0.7 35.5 18.4 14.5 25.4 0.0 16.09 96.154951 9.4 0 2015 6 29 20:20:00
3 2015-06-29 21:20:00 0.0 1.3 1013.678596 55.6 0.0 23.2 -0.9 35.5 18.4 13.8 23.2 0.0 16.09 95.155405 6.8 0 2015 6 29 21:20:00
4 2015-06-29 22:20:00 0.0 0.9 1013.606006 59.1 0.0 22.6 -0.7 35.5 18.4 14.2 22.6 0.0 16.09 94.083056 5.8 0 2015 6 29 22:20:00
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
1223615 2022-06-30 19:20:00 0.0 2.5 1011.700000 51.4 0.0 24.9 -0.3 32.0 17.0 14.2 24.9 0.0 13.70 50.000000 5.8 19 2022 6 30 19:20:00
1223616 2022-06-30 20:20:00 0.0 3.7 1013.100000 61.8 0.0 23.1 -0.2 32.0 17.0 15.4 23.1 0.0 13.59 50.000000 7.9 19 2022 6 30 20:20:00
1223617 2022-06-30 21:20:00 0.0 3.8 1014.300000 72.2 0.0 21.4 -0.5 32.0 17.0 16.2 21.4 0.0 13.46 50.000000 8.3 19 2022 6 30 21:20:00
1223618 2022-06-30 22:20:00 0.0 3.1 1015.000000 79.8 0.0 20.8 -0.2 32.0 17.0 17.2 20.8 0.0 11.85 30.000000 9.0 19 2022 6 30 22:20:00
1223619 2022-06-30 23:20:00 0.0 1.9 1015.200000 90.0 0.0 20.3 0.1 32.0 17.0 18.6 20.3 0.0 8.05 40.000000 10.8 19 2022 6 30 23:20:00

1223620 rows × 21 columns

The predictions have been made correctly because there are no longer any missing values.

In [18]:
METEO_PROPHET.groupby('Year')[columns_with_missing_data_METEO].apply(lambda x: x.isnull().sum())
Out[18]:
precip1Hour pressureChange pressureMeanSeaLevel snow1Hour temperature temperatureChange24Hour temperatureMax24Hour temperatureMin24Hour temperatureDewPoint temperatureFeelsLike uvIndex visibility windDirection
Year
2015 0 0 0 0 0 0 0 0 0 0 0 0 0
2016 0 0 0 0 0 0 0 0 0 0 0 0 0
2017 0 0 0 0 0 0 0 0 0 0 0 0 0
2018 0 0 0 0 0 0 0 0 0 0 0 0 0
2019 0 0 0 0 0 0 0 0 0 0 0 0 0
2020 0 0 0 0 0 0 0 0 0 0 0 0 0
2021 0 0 0 0 0 0 0 0 0 0 0 0 0
2022 0 0 0 0 0 0 0 0 0 0 0 0 0

Let's look at the distribution of the variables that had missing data before and after imputation to see if there is anything unusual that we need to address.

In [19]:
for col in columns_with_missing_data_METEO:
    if (METEO[col].min() == 0) & (METEO_PROPHET[col].min() < 0):
        METEO_PROPHET[col] = METEO_PROPHET[col].apply(lambda x: 0 if x < 0 else x)

    plt.figure(figsize=(8,4))
    sns.histplot(x=METEO[col], kde=True)
    sns.histplot(x=METEO_PROPHET[col], kde=True)
    plt.show()

We see that "uvIndex" has values that it should not be able to have and we are going to deal with them

In [20]:
METEO_PROPHET['uvIndex'].unique()
Out[20]:
array([ 0.        ,  2.        ,  3.        ,  5.        ,  8.        ,
        9.        ,  7.        ,  4.        ,  1.        ,  6.        ,
       10.        ,  1.33979689,  1.35685186,  1.38915057,  1.35306158,
        1.37437932,  1.35675337,  1.33734155,  1.35933897,  1.35340824,
        1.35012993,  1.34153404,  1.38170465,  1.34854181,  1.36749344,
        1.35499112,  1.36305064,  1.36210725,  1.35616879,  1.35279281,
        1.35541468])

To do this, we are going to iterate the rows where "uvIndex" takes values that should not be and replace it with the fashion of values of "uvIndex" that it has had in the same month at the same time.

In [21]:
for i, row in METEO_PROPHET[~METEO_PROPHET['uvIndex'].isin(METEO['uvIndex'].unique())].iterrows():
    METEO_PROPHET.loc[i,'uvIndex'] = METEO_PROPHET[(METEO_PROPHET['Month'] == row['Month']) & (METEO_PROPHET['Time'] == row['Time'])]['uvIndex'].mode()[0]
    
METEO_PROPHET['uvIndex'].unique()
Out[21]:
array([ 0.,  2.,  3.,  5.,  8.,  9.,  7.,  4.,  1.,  6., 10.])

After this we can see that it no longer takes strange values and that the distribution of the variable is similar to the distribution prior to imputation.

In [22]:
plt.figure(figsize=(8,4))
sns.histplot(x=METEO['uvIndex'], kde=True)
sns.histplot(x=METEO_PROPHET['uvIndex'], kde=True)
plt.show()

We are going to finalise this dataset by reducing it to daily information, increasing the number of variables to have the information of the mean, median, mode, minimum, maximum of the variables but grouping it by day.

In [27]:
METEO_PROPHET
Out[27]:
validTimeUtc precip1Hour pressureChange pressureMeanSeaLevel relativeHumidity snow1Hour temperature temperatureChange24Hour temperatureMax24Hour temperatureMin24Hour temperatureDewPoint temperatureFeelsLike uvIndex visibility windDirection windSpeed ID_ESTACION Year Month Day Time
0 2015-06-29 18:20:00 0.0 -0.1 1013.793693 33.2 0.0 31.9 -0.2 35.5 18.4 13.7 31.9 0.0 16.09 97.653293 15.5 0 2015 6 29 18:20:00
1 2015-06-29 19:20:00 0.0 0.5 1013.778149 39.4 0.0 29.5 -0.1 35.5 18.4 14.3 29.7 0.0 16.09 96.926700 13.7 0 2015 6 29 19:20:00
2 2015-06-29 20:20:00 0.0 1.2 1013.737503 50.9 0.0 25.4 -0.7 35.5 18.4 14.5 25.4 0.0 16.09 96.154951 9.4 0 2015 6 29 20:20:00
3 2015-06-29 21:20:00 0.0 1.3 1013.678596 55.6 0.0 23.2 -0.9 35.5 18.4 13.8 23.2 0.0 16.09 95.155405 6.8 0 2015 6 29 21:20:00
4 2015-06-29 22:20:00 0.0 0.9 1013.606006 59.1 0.0 22.6 -0.7 35.5 18.4 14.2 22.6 0.0 16.09 94.083056 5.8 0 2015 6 29 22:20:00
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
1223615 2022-06-30 19:20:00 0.0 2.5 1011.700000 51.4 0.0 24.9 -0.3 32.0 17.0 14.2 24.9 0.0 13.70 50.000000 5.8 19 2022 6 30 19:20:00
1223616 2022-06-30 20:20:00 0.0 3.7 1013.100000 61.8 0.0 23.1 -0.2 32.0 17.0 15.4 23.1 0.0 13.59 50.000000 7.9 19 2022 6 30 20:20:00
1223617 2022-06-30 21:20:00 0.0 3.8 1014.300000 72.2 0.0 21.4 -0.5 32.0 17.0 16.2 21.4 0.0 13.46 50.000000 8.3 19 2022 6 30 21:20:00
1223618 2022-06-30 22:20:00 0.0 3.1 1015.000000 79.8 0.0 20.8 -0.2 32.0 17.0 17.2 20.8 0.0 11.85 30.000000 9.0 19 2022 6 30 22:20:00
1223619 2022-06-30 23:20:00 0.0 1.9 1015.200000 90.0 0.0 20.3 0.1 32.0 17.0 18.6 20.3 0.0 8.05 40.000000 10.8 19 2022 6 30 23:20:00

1223620 rows × 21 columns

In [28]:
METEO_FINAL = (METEO_PROPHET.groupby(['ID_ESTACION', 'Day', 'Month', 'Year'], sort = False)
                               .agg({'precip1Hour':[np.nanmean,np.nanmin,np.nanmax,np.nansum],
                                     'pressureChange':[np.nanmean,np.nanmin,np.nanmax,np.nanmedian],
                                     'pressureMeanSeaLevel':[np.nanmean,np.nanmin,np.nanmax],
                                     'relativeHumidity':[np.nanmean,np.nanmin,np.nanmax,np.nanmedian],
                                     'snow1Hour':[np.nanmean,np.nanmin,np.nanmax,np.nansum],
                                     'temperature':[np.nanmean,np.nanmax,np.nanmin,np.nanmedian],
                                     'temperatureDewPoint':[np.nanmean,np.nanmin,np.nanmax,np.nanmedian],
                                     'temperatureFeelsLike':[np.nanmean,np.nanmin,np.nanmax,np.nanmedian],
                                     'uvIndex':[lambda x: mode(x)[0], np.nanmax, np.nanmin],
                                     'visibility':[np.nanmean,np.nanmin,np.nanmax],
                                     'windDirection':[np.nanmean,np.nanmedian],
                                     'windSpeed':[np.nanmean,np.nanmin,np.nanmax,np.nanmedian]
                                          })
                               .rename(columns={'<lambda_0>': 'nanmode', '<lambda>': 'nanmode'}))

METEO_FINAL.columns = ["_".join(col) for col in METEO_FINAL.columns.values]
METEO_FINAL.columns = [i.split("_")[1][3:] + i.split("_")[0].capitalize().replace("1hour","") for i in METEO_FINAL.columns.values] 
METEO_FINAL = METEO_FINAL.reset_index()
METEO_FINAL
Out[28]:
ID_ESTACION Day Month Year meanPrecip minPrecip maxPrecip sumPrecip meanPressurechange minPressurechange maxPressurechange medianPressurechange meanPressuremeansealevel minPressuremeansealevel maxPressuremeansealevel meanRelativehumidity minRelativehumidity maxRelativehumidity medianRelativehumidity meanSnow minSnow maxSnow sumSnow meanTemperature maxTemperature minTemperature medianTemperature meanTemperaturedewpoint minTemperaturedewpoint maxTemperaturedewpoint medianTemperaturedewpoint meanTemperaturefeelslike minTemperaturefeelslike maxTemperaturefeelslike medianTemperaturefeelslike modeUvindex maxUvindex minUvindex meanVisibility minVisibility maxVisibility meanWinddirection medianWinddirection meanWindspeed minWindspeed maxWindspeed medianWindspeed
0 0 29 6 2015 0.0 0.0 0.0 0.0 0.650000 -0.1 1.3 0.70 1013.685537 1013.519275 1013.793693 49.133333 33.2 59.1 53.25 0.0 0.0 0.0 0.0 25.683333 31.9 21.5 24.30 13.833333 12.5 14.5 14.00 25.716667 21.5 31.9 24.30 0.0 0.0 0.0 16.090000 16.09 16.09 95.526197 95.655178 9.433333 5.4 15.5 8.10
1 0 30 6 2015 0.0 0.0 0.0 0.0 -0.320833 -1.5 1.4 -0.45 1013.513003 1013.149230 1013.802098 45.441667 21.6 71.1 41.25 0.0 0.0 0.0 0.0 26.275000 35.7 17.5 26.40 12.129167 10.4 14.1 11.75 26.304167 17.5 35.7 26.55 0.0 9.0 0.0 16.090000 16.09 16.09 93.747468 93.253171 10.762500 3.6 20.2 9.15
2 0 1 7 2015 0.0 0.0 0.0 0.0 0.087500 -0.8 1.7 -0.20 1013.754212 1013.338493 1014.073722 53.762500 34.7 69.4 54.00 0.0 0.0 0.0 0.0 24.400000 31.0 18.3 24.20 13.950000 12.2 16.0 13.80 24.566667 18.3 31.0 24.20 0.0 10.0 0.0 16.090000 16.09 16.09 88.922818 89.563662 12.770833 3.6 24.1 11.35
3 0 2 7 2015 0.0 0.0 0.0 0.0 0.400000 -0.6 1.7 0.40 1013.759118 1013.464394 1013.983624 64.537500 50.2 82.0 63.65 0.0 0.0 0.0 0.0 23.591667 28.5 19.8 23.35 16.325000 13.9 18.9 16.45 23.958333 19.8 29.6 23.35 0.0 9.0 0.0 16.090000 16.09 16.09 91.877629 91.866733 10.966667 4.0 18.7 10.10
4 0 3 7 2015 0.0 0.0 0.0 0.0 0.150000 -0.8 1.7 0.05 1013.702045 1013.386914 1013.944178 70.408333 45.1 96.2 73.50 0.0 0.0 0.0 0.0 24.245833 29.0 20.8 23.05 17.954167 15.4 20.2 18.00 24.625000 20.8 30.0 23.05 0.0 10.0 0.0 16.090000 16.09 16.09 83.800562 83.825151 14.683333 7.6 23.0 14.20
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
51175 19 26 6 2022 0.0 0.0 0.0 0.0 0.062500 -1.9 2.5 -0.10 1009.804167 1008.000000 1011.300000 59.458333 32.4 79.8 64.75 0.0 0.0 0.0 0.0 23.029167 30.9 15.2 22.50 13.912500 10.5 16.3 14.00 23.154167 15.2 30.9 22.50 0.0 9.0 0.0 13.361250 12.13 15.06 80.833333 60.000000 7.454167 2.5 13.7 8.25
51176 19 27 6 2022 0.0 0.0 0.0 0.0 1.012500 -0.4 3.3 0.85 1014.675000 1011.100000 1019.800000 76.350000 57.4 95.1 78.00 0.0 0.0 0.0 0.0 21.445833 25.4 17.6 21.25 16.837500 15.8 18.2 16.70 21.445833 17.6 25.4 21.25 0.0 9.0 0.0 11.267500 8.05 14.03 34.583333 35.000000 11.941667 2.9 17.6 13.15
51177 19 28 6 2022 0.0 0.0 0.0 0.0 -0.229167 -2.1 1.9 -0.10 1017.529167 1015.100000 1019.300000 60.425000 27.8 100.0 57.10 0.0 0.0 0.0 0.0 23.570833 31.6 15.8 23.35 13.979167 9.7 16.5 14.50 23.612500 15.8 31.6 23.35 0.0 9.0 0.0 11.984167 8.05 14.94 161.666667 190.000000 8.666667 3.6 16.6 7.40
51178 19 29 6 2022 0.0 0.0 0.0 0.0 -0.600000 -2.1 1.8 -0.90 1013.695833 1010.500000 1016.900000 51.450000 24.7 81.8 49.05 0.0 0.0 0.0 0.0 24.766667 32.6 16.7 24.40 12.775000 9.0 15.2 12.95 24.804167 16.7 32.6 24.40 0.0 9.0 0.0 13.778750 12.43 15.41 189.166667 190.000000 11.016667 4.0 20.2 10.40
51179 19 30 6 2022 0.0 0.0 0.0 0.0 0.241667 -1.3 3.8 -0.25 1011.770833 1009.400000 1015.200000 54.387500 28.0 90.0 51.40 0.0 0.0 0.0 0.0 24.737500 32.3 17.2 24.05 13.737500 10.5 18.6 13.65 24.812500 17.2 32.3 24.05 0.0 9.0 0.0 13.325000 8.05 15.24 114.583333 100.000000 6.808333 2.9 11.2 6.15

51180 rows × 47 columns

Let's visualise the distribution of all variables in the resulting dataset

In [29]:
variable_columns = {
    'precip': ['meanPrecip', 'minPrecip', 'maxPrecip', 'sumPrecip'],
    'pressureChange': ['meanPressurechange', 'minPressurechange', 'maxPressurechange'],
    'pressureMeanSeaLevel': ['meanPressuremeansealevel', 'minPressuremeansealevel', 'maxPressuremeansealevel'],
    'relativeHumidity': ['meanRelativehumidity', 'minRelativehumidity', 'maxRelativehumidity'],
    'snow': ['meanSnow', 'minSnow', 'maxSnow'],
    'temperature': ['meanTemperature', 'minTemperature', 'maxTemperature'],
    'temperatureDewPoint': ['meanTemperaturedewpoint', 'minTemperaturedewpoint', 'maxTemperaturedewpoint'],
    'temperatureFeelsLike': ['meanTemperaturefeelslike', 'minTemperaturefeelslike', 'maxTemperaturefeelslike'],
    'uvIndex': ['modeUvindex', 'maxUvindex', 'minUvindex'],
    'visibility': ['meanVisibility', 'minVisibility', 'maxVisibility'],
    'windDirection': ['meanWinddirection'],
    'windSpeed': ['meanWindspeed', 'minWindspeed', 'maxWindspeed']
}

statistics = ["max", "mean", "min", 'mode', 'sum', 'median']
fig, axes = plt.subplots(nrows=len(variable_columns), ncols=len(statistics), figsize=(20, 35)
)


for i_row, (axes_row, variable) in enumerate(zip(axes, variable_columns.keys())):
    for ax, stat in zip(axes_row, statistics):
        meteo_column = stat + variable.title()
        
        try:
            data = METEO_FINAL[meteo_column]
        except KeyError:
            ax.axis("off")
            continue
            
        sns.histplot(data=data, kde=True, ax = ax)
        ax.set_title(meteo_column)
        ax.set_xlabel("Value")
        ax.set_ylabel("Frequency")
        

plt.tight_layout()
plt.show()
In [30]:
METEO_FINAL.to_csv('./Datos_Completos/DATOS_METEO_FINAL.csv', index=None)